1. Field of the Invention
The present invention generally concerns the computerized composition and display of photorealistic perspective images of real or virtual three-dimensional objects, most typically objects for sale, located in, surrounding or in front of, real or virtual three-dimensional scenes where the objects are not fanciful, and the scenes are generally not fanciful, but are or could be existing within the real world, for the purposes of (i) promoting or selling the depicted objects directly to a consumer, and/or (ii) generating advertising images of the objects.
The composition and display may be, by way of example, the selection and placement of furniture and furnishings within interior rooms for purposes of promoting or selling such furniture and furnishings as depicted in photorealistic three-dimensional scenes chosen or specified by and potentially unique to the viewer, such as in the viewer's own home or a scene for an image to be used in an advertisement.
The present invention particularly concerns the communications-network-linked software-program-driven computerized selection, composition and photorealistic imaging of objects such as furniture sand furnishings, including virtual objects that are not yet fabricated, in perspective view and accurately three-dimensionally positioned within real or virtual rooms or scenes responsively to interaction with professional interior designers decorators or sales associates, and/or generators of advertising images; the man-machine interaction being so that designs of, inter alia, (i) furnished rooms may be both interactively developed across a digital communications network and, ultimately, displayed to a prospective consumer-purchaser of the room furnishings and/or (ii) photorealistic advertising images may be interactively developed remotely from any such graphics rendering system as has available the desired objects and textures and is capable of rendering such photorealistic images, and then transmitting them across a digital communications network, the development using only computer graphics objects, textures, maps and other such input to computer-based image rendering systems, some of which may have been derived from photographic sources.
The present invention also particularly concerns communications-network-linked software-program-driven computerized selection, composition and display—in response to interaction with professional designers and creative advertising copyists—of photorealistic three-dimensional renderings of goods suitable for presentation directly from the network, or indirectly such as via a print medium, to a potential consumer(s) of the goods, and methods of doing business based on the generation and presentation of such photo-realistic images.
2. Background
2.1 General Background
The present invention will be seen to have both (i) business and (ii) technical aspects. Technically, the present invention will be seen to broadly concern a software-based system for both (i) interactively producing, and (ii) rendering, across a digital communications network, photo-realistic composite images of interactively customized products in such surroundings as are customary to and specified by the viewer of the image, different viewers desiring and appreciating different surroundings. Business-wise, this rendering, typically conducted as a network service at no cost to enrolled business professionals, is for the purpose of promoting and selling such products. These customary surroundings are referred to herein as “background scenes”. The selected and placed product images will be seen to include, by way of example, images of interior furnishings, rendered within background scenes of interior spaces.
2.2 Previous Patents Directed to Interior or Exterior Design
U.S. Pat. No. 4,318,121 to Taite, et. al., for INTERIOR DECOR COMPOSITION AND DISPLAY SYSTEMS concerns display systems are of particular utility in assisting interior decorators and homemakers in selecting appropriate color and fabric combinations. In particular, the subject display systems provide a luminous image of an interior decor including components, such as furniture, drapes and carpets. The system picks up colors from different samples, each of which typically has an area smaller than the area of any displayed component. These picked-up colors are selectively filled in the displayed components when the image is displayed. Color arrangements are changed by substituting other picked-up colors in the display of particular components.
U.S. Pat. No. 4,931,929 to Sherman for DESIGN COMPONENT SELECTION COMPUTER WITH SPECIFICATION OF PRODUCT CHARACTERISTICS AND OF COLOR BY MACHINE READABLE DEVICE concerns a process for identification, description and display of design components, such as interior decoration products, selected by describing desired component characteristics to a digital computer. An image database is provided, preferably in the form of an addressable optical videodisk, each frame of which portrays an individual design component, and the images are displayed by operation of a computer-controlled archive system such as a random-access optical videodisk player. A characteristics database is provided to characterize each portrayed design component by a variety of product categories including at least color, price, manufacturer and image database address. The process obtains one or more desired component characteristics, examines the characteristics database to identify design components meeting the desired product characteristics, and displays the identified component images together with a complete record of characteristics for each product. The desired color characteristic can be obtained by selection from a text dictionary of color identification terms, by machine inspection of a machine-readable color comparison sample, or by optical spectrum analysis of a pre-existing product sample.
U.S. Pat. No. 5,255,352 to Falk for MAPPING OF TWO-DIMENSIONAL SURFACE DETAIL ON THREE-DIMENSIONAL SURFACES concerns a system and method for providing surface detail to mathematically-defined three-dimensional surfaces which preserves the specific dimensional integrity of the surface detail image being mapped in order to provide dimensionally correct surface detail. This is accomplished by performing the intermediate step of mapping the surface detail image to a two-dimensional flattened pattern piece representation of the surface and thereafter mapping this representation to the three-dimensional surface. The system and method of the present invention may be used as one function of a greater three-dimensional computer aided design system for any industry which utilizes flat sheets of material formed or assembled into a finished product, including textiles, apparel, footwear, upholstered furnishings, automotive or interior design.
U.S. Pat. No. 5,601,431 to Howard for an INTERIOR DESIGN SYSTEM AND METHOD concerns an interior design system comprising a display assembly having three flexible transparent overlays, a plain background sheet hinged to corresponding edges of the overlays, and a template sheet hinged to a free edge of the background sheet and having “scale” background and foreground grid sections. All this is for assisting in laying out interior room decorator's designs with a comprehensive catalog of design elements printed with scale designs of perspective furniture and accessory designs, and architectural elements for creating the room space to be decorated. The catalog pages have rows of printed scale designs covered with transparent plastic, with overlying, selectively removable and replaceable design elements. The design elements have designs that are printed to the same scale on carrier sheets of flexible “static-cling” material. These carrier sheets are rectangular sections that are cut from a printed sheet the same size and shape as the backing sheet. In this manner, the catalog pages provide marked storage positions for the design elements after use.
In addition, the backing sheets are marked with descriptive labels and special coded symbols in distinctive shapes and colors for assisting in selection and coordination of interior designs, and also are imprinted with generic labels for the designs. The method constitutes the steps of (i) providing the catalog of scale images and the background and foreground grid sections, (ii) labeling and coding the images for identification, coordination and storage, (iii) selecting candidate images from the catalog and transferring them to the grid sections, (iv) applying the images in the desired positions and combinations, thereby to create a decorator's design, plus, optionally, (v) applying colors as an overlay on the completed design for a more complete rendering.
U.S. Pat. No. 5,940,806 to Danial for a METHOD AND APPARATUS FOR RECEIVING, STORING AND PROVIDING DATA CORRESPONDING TO GEOMETRICALLY DESCRIBABLE RAW MATERIALS concerns an apparatus and method employing a programmable computer for providing a list of data items corresponding to geometrically describable materials based upon a profile specified by a user. The list of data items is stored in a data base. Each data item includes geometrically descriptive terms and trading parameters identifying characteristics of an item offered for sale by an advertiser. A profile is generated by a user which includes geometrically descriptive terms and trading parameters identifying characteristics of an item sought for purchase by the user. The profile identified by the user is compared to each data item stored in the data base and data items are identified which match the profile. The data item identified are provided to the user in accordance with the user's specific delivery instructions.
U.S. Pat. No. 6,005,969 to Thomas, et. al., concerns METHODS AND SYSTEMS FOR MANIPULATION OF IMAGES OF FLOOR COVERINGS OR OTHER FABRICS that enable a highly streamlined and efficient fabric or textile sampling and design process. The sampling and design process is claimed to be particularly valuable in the design and selection of floor coverings, wall coverings and other interior design treatments. A digital library of fabric models is created, preferably including digitized full-color images associated with digital representations of positions that are located within, and which characterize, the models. Via an application implemented according to conventional software methods and running on conventional hardware having high resolution graphics-processing capabilities, a user may navigate among the set of alternative models, and may modify the positions of the selected models to test out desired combinations of characteristics—such as poms or yarn ends, or models of floor coverings—and view the results in high resolution. A method is provided for substituting colors in digital images of photographic quality, while preserving their realism particularly in the vicinity of shadows. The resulting samples or designs can be stored and transmitted over a telecommunications network or by other means to a central facility that can either generate photographic-quality images of the samples, or can directly generate actual samples of the carpet or other material of interest.
2.3 A Specific Previous Interior Design System Operative on a Private Satellite Down-link, Phone Line Up-link Communications Network
The system and method of the present invention will be seen to involve interactive communications across a digital communications network for the purpose of producing images that typically include furniture. At least one previous upholstered furniture customization system involves a real-time network-based interactive system for support of visualizing upholstered furniture prior to placing an order for the upholstered furniture.
This existing interactive system of La-Z-Boy™ Incorporated, and possibly others, is available for use by sales associates in La-Z-Boy™ furniture showrooms. The system permits prospective customers to preview La-Z-Boy furniture upholstered in La-Z-Boy offered fabrics. It requires a La-Z-Boy™ furniture item number, and a La-Z-Boy™ fabric number. The system then maps the fabric onto a static perspective view of the selected furniture item.
To the extent of depicting furniture, this previous system is similar to the system and method of the present invention. However, the system used by La-Z-Boy™ is considerably different from the system of the present invention as to each of, inter alia, (i) system interaction (rote versus sophisticated) with the targeted user (a furniture salesman versus an interior designer), (ii) how the furniture images are generated and presented (2D versus 3D, and fixedly versus being shown in context), (iii) the technical capabilities of the image generation (no provision for scaling, rotation and placement of upholstered furniture images into background scenes), and (iv) the business model supported (retail store furniture promotion vs. a professional-level interior design tool supporting furniture selection, placement and procurement as well as promotion.)
2.4 A Specific Interior Design System Operative on the Internet
The system and method of the present invention will be seen to involve communications across a digital communications network, to wit: the Internet. At least one interactive design system—evolved as a real-time network-based evolution of an older PC-based software program product—already (circa 2000) exists on the Internet. This existing system is superficially similar to the system and method of the present invention. However, the existing system is again considerably different from the system of the present invention as to each of, inter alia, (i) the targeted audience (amateurs versus professionals), (ii) how the images are generated and presented (2D versus 3D), and (iii) the business model supported (furniture and software promotion vs. professional-level design assistance, furniture promotion and procurement.)
The existing interactive system is the called the cMyVision™ Free Internet design software for use in interior, exterior, home and landscape improvement. The cMyVision™ software is a product, and cMyVision™ web site a service, of Visual Applications, Inc. (VisApp), headquartered in Kansas City, Mo. (cMyVision™ is a trademark of VisApp.) VisApp is a publisher of personal visualization technology, with a web site at <http://www.visapp.com>of the year 2000.
Use of cMyVision™ Free Internet design software for interior, exterior, home and landscape improvement initially requires a digital photo of a house, normally the client's own house. Into this 2D scene a 2D object is inserted from the cMyVision library, or database. The system of the present invention will be seen to differ in that, from two or more 2D views, a 3D model of the client's 3D space, replete with dimensions, is first constructed. Then this 3D space is populated with 3D objects such as furniture. Accordingly, the cMyvision system operates to produce two-dimensional (2D) images while the system of the present invention, employing full three-dimensional (3D) models of both scenes and objects, produces full perspective scene views.
VisApp also sells a product called VisualPhile™ Personal Visualization Software, touted to support professional interior remodeling and design.
VisualPhile™ is personal computer software with the same features as cMyvision™. It runs on a personal computer without an Internet connection, as its library is included with the program. It is primarily suited to providing photo-realistic visualizations of home remodeling as opposed to decorating projects. As in cMyVision™, a certain degree of photo-realism stems from the use of photographs for both rooms and built-ins, appliances and furnishings.
However, there exist inherent limitations—arguably severe when applied to professional interior design—in both cMyVision™ sand VisualPhile™ and their use of digital photographs. These previous systems (i) are not based upon 3D coordinates (as will be seen to be used in the present invention), so furniture images have to be placed and scaled visually by the user; and (ii) the furniture objects are 2D images, so they cannot be rotated to match the angle of the room photograph (as will seen to be the case in the present invention). Therefore, in both cMyVision™ and VisualPhile™, room photos must be taken from a specified perspective, as their furniture photographs are all taken from this angle. Moreover, (iii) because the furniture images are only photographs, and not 3D models, it is not possible to map fabrics on them. This last limitation cannot be mitigated and is highly significant in professional interior design applications.
2.5 Stand Alone Personal Computer Interior Design Programs
A number of software programs are commercially available which support interior design. These programs generally run on personal computers and allow the user to design rooms, place furniture objects into rooms, change wall colors and furniture fabrics, and display the customized furniture in the decorated rooms. Custom Home 3-D Design and Décor, published by Sierra On-line, Inc., 5 in 1 Home Design by Punch! Software LLC, and 3D Home Interiors by Broderbund all support most of these design-assistance and visualization objectives. In addition, numerous commercially-available programs support architectural rendering, including computer-aided design (CAD) programs like AutoCAD, which incorporate 3-D functionality and therefore have application to interior design.
Custom Home 3-D Design and Décor includes a program called Photo Home Interiors™. This program, unlike the others, allows users to import photographs of actual rooms into the program, and then place furniture objects into the room images. The objects include 3-D furniture models, which can be rotated and scaled to fit the scene. However, the room images are not associated with a 3-D model of the room. In other words, while Photo Home Interiors™ does allow furniture objects to be shown as an overlay on top of room photographs, it lacks the capability of rendering furniture objects placed onto floor plans in photographically-derived scenes. Therefore, this program does not allow wall and floor coverings to be displayed in the proper perspective. In addition, the 3-D objects need to be visually scaled by the user to compensate for the perceived distance from the camera position. With this system, it is not possible to evaluate furniture objects of specific dimensions as to scale. For all of these reasons, is cumbersome to use on a personal computer, and would be even more so over a digital communications network.
Furthermore, the furniture objects are generic and therefore not available for purchase, no procurement features are available, the furniture and fabric database is very limited in quantity and quality. Finally, this program does not operate over a digital network.
As a result of such limitations, even this program has limited professional application. These limitations in the prior art are overcome in the present invention, which incorporates proprietary technology developed to support this highly desirable functionality.
2.6 Existing 3D Image Capture Technology
The present invention will be seen to beneficially employ in the generation of three-dimensional (3D) images of a human head a 3D image capture technology. Such a technology is commercially available, circa 2000, from Minolta Co., Ltd. in combination with MetaCreations, Inc., in the 3D 1500 camera product. The 3D 1500 is a complete hardware/software package that combines a quality Minolta digital camera with MetaCreation's MetaFlash 3D image production technology. It allows creation of high-quality 3D models in MetaCreation's MetaStream file format, which can be transmitted quickly over the network for further manipulation.
Another example of a three-dimensional (3D) image acquisition system for deriving 3D models of the human head is the ShapeSnatcher 3D-Toolkit from Eyetronics, Inc. After calibration, data can be captured within the time needed for a single photograph or frame. The 3D model is textured, with a perfect alignment between the shape and the texture. Skin color, marks or scars are visible as they would be on a photograph.
2.7 Relationship to the Predecessor Patent Application
The present patent application, a continuation-in-part of U.S. patent application Ser. No. 09/643,507 filed on Aug. 22, 2000, may be perceived as relating to that application for being the substantial equivalent thereof, incorporating only a somewhat narrow change. Namely, a change is made in (i) what software-based functions are performed at the computer of a (network-connected) user/viewer/client/prospective purchaser, and (ii) what particular display is preliminarily visible to the user/viewer/client/prospective purchaser (at a time before a photorealistic 3-D image of selected, and selectively positioned and selectively illuminated, objects is centrally generated, and communicated across the communications network to the user/viewer/client/prospective purchaser). Namely, the predecessor application discusses “object based rules” and “2D to 3D translation” in that a (top) plan view of a room (or the like) showing the locations and rotations of scene objects (typically furniture) is combined with object-based rules (i.e., chairs always stand on their legs on the floor) to generate a scene file suitable for a high-powered graphics server to produce a photorealistic rendering of 3D objects in 3D scenes.
The present invention, and application, in no way repudiates this earlier approach, but is simply a variant thereof. Namely, the user/viewer/client/prospective purchaser constructs (i) by (conventional, circa 2002) graphics software process at his/her computer, and (ii) by use of miniature “stand-in” models and textures that represent items that will ultimately be rendered with photo-realism on a remote rendering system, a scene of cartoonish-like, even stick-figure, models and textures, located within a room or other scene. The user/viewer/client/prospective purchaser can thus preview and adjust a perspective view—replete with lighting, and sometimes even including color(s) and/or texture(s)—of the scene, rather than just a plan view of the layout of the objects within the scene. Such adjustments to the preview view would include changes to the angle of view, the location, height and direction of the camera, the position of lights and illumination levels.
Clearly the fundamental aspect of the previous invention to the effect that (i) graphics manipulation tractable, and suitable, to the skills and the software and the power of a user computer should be done at the user computer, while (ii) photo-realistic scene rendering in done on a server (or allied) computer, is not changed in the present invention. Clearly the fundamental aspect of the previous invention to ultimately render a photo-realistic image of selected items in situ is not changed in the present invention.
In a broader view, the present invention and application may be perceived to validate the former invention and application. Namely, even if both applications arise from the same source—which they do—the scope of the method of both inventions of photo-realistically displaying goods selected by a user/viewer/client/prospective purchaser in situ an environment also selected or specified by the user/viewer/client/prospective purchaser is broadened when it is realized that it is not the details of the process—presenting a plan image versus a low resolution 3D perspective view image to the user/viewer/client/prospective purchaser—that are important, but the overall flow of information, and the overall generation of a photorealistic image using computational resources of the user/viewer/client/prospective purchaser to “frame” the scene, and the greater computational resources of a server and allied computers to, ultimately, render the 3D objects in a 3D scene with the full photorealism that is perceived to be required for an optimally informed purchasing decision.
2.9 Web 3D
The present invention will be seen to use at client computers an ordinary, low level, graphics capability that is, circa 2002, increasingly being built into browser and browser plug-ins.
One leading standard for the generation of 3D images over the Internet is Web3D. In applications such as that of the present invention a preview that can be directly based on Web3D permits a preview of a scene. A Web3D application may, and often does, present to the user controls for placing and adjusting objects (in the broad sense of this word, including scenes), and selecting textures. This will be seen to be of paramount importance in the use of the present invention in the development of photorealistic images, including for advertising.
Accordingly, as background to the present invention, the Web3D Consortium, Inc.—expounded at <www.web3d.org>—is a nonprofit corporation with the mission of fostering and evangelizing all three-dimensional technologies on the Internet. The Web3D Consortium's goals are to: (1) foster the ongoing development of three dimensional specifications and standards on the Internet, (2) promote rapid industry adoption of and adherence to the X3D specification; (3) offer opportunities for the three-dimensional development community to meet and cooperate in the evolution of Web3D technologies; (4) educate the business and consumer communities on the value, benefits and applications of three dimensional technologies on the Internet; (5) support the creation of conformance tests to assure Web3D interoperability; and (6) liaison with educational institutions, government research institutes, technology consortia, and other organizations which support and contribute to the development of specifications and standards for Web3D.
The Web3D Consortium was formed to provide a forum for the creation of open standards for Web3D specifications, and to accelerate the worldwide demand for products based on these standards through the sponsorship of market and user education programs. Web3D applications have been actively pursued by many organizations for quite some time. This community has spearheaded the development of the VRML 1.0 and 2.0 specifications, which provide the basis for the development of associated applications. The organizations involved in this effort felt that the creation of an open consortium focused exclusively on Web3D would provide the structure necessary to stabilize, standardize, and nurture the technology for the entire community.
Today, year 2002, the Web3D Consortium is utilizing its 1500-2000 strong Internet development community, and its broad-based industry support to systematically move the VRML97 ISO Standard forward. Its many prominent technical activities include the Extensible 3D (X3D) specification, which is extending VRML97, using the Extensible Markup Language (XML). Through the well-coordinated efforts of dedicated Working and Task Groups and ISO/W3C Liaisons, the Web3D Consortium is maintaining and extending its standardization activities well into the next Millennium.
There are numerous sources of information on authoring Web3D content for both programmers and, increasingly, designers.
A Web3D Browser is a piece of software that lets you view and navigate around a 3D scene over the Web, for example a VRML file. These browsers can either be stand alone programs or, more typically, they are plug-ins for common Web browsers such as Netscape Communicator or Internet Explorer. A number of companies produce Web3D products. A few of the common systems in use circa 2002 include the following: Actify 3DView, Conmtact from Blaxxun Interactive; Cortona from ParallelGraphics; Cosmo Player from Platinum Technologies; Fraunhofer Institute CASUS Presenter; Graz University VRWave; Java X3D Browser; OpenWorlds Horizon Browsers; Shout3d from Eyematic; Sony Community Place; Superscape Viscape Support; UpperCut Software WorldProbe. A more complete list of Web3D browser software can be found at the aforementioned web3D web site.